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is displayed to the^^lgast one-user. The customized pre- 
sentation is modified based on input from the at'least one 
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SYSTEM AND METHODS FOR AN 
ARCHITECTURAL FRAMEWORK FOR 
DESIGN OF AN ADAPTIVE, 
PERSONALIZED, INTERACTIVE CONTENT 
DELIVERY SYSTEM 

CROSS-REFERENCE TO RELATED 
APPLICATIONS 

This invention claims the benefit of U.S. Provisional 
Application No. 60/099,947, filed on Sep. 11, 1998, the 
contents of which is expressly incorporated by reference 
herein. 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

Ihis invention relates to architectural frameworks for 
development of multimedia applications, and more specifi- 
cally to architectural frameworks for developing adaptive, 
personalized, interactive multimedia applications and ser- 20 
vices. 

2. Background and Material Information 

In general, designing and implementing interactive sys- 
tems is a complex and lengthy task. If one adds multimedia 
to the development equation, the level of complexity, the 
content variability and the required management support 
immediately soars and can overwhelm the development 
process. On the other hand, there presently exists a very 
dynamic and rich environment that potentially offers a 3Q 
business opportunity allowing one to build a family of 
applications that can be strongly differentiated by leveraging 
the same rich and complex content. Thus, a double edged 
sword exists. 

If one examines the requirements, present and future, of 35 
information, more specifically multimedia information, one 
discovers that in general these requirements are a response 
to the "dynamics of information". These dynamics can be 
characterized by: constantly changing information; broad 
user population; and heterogenous landscape of delivery ^ 
devices. If one grafts onto this picture the dynamics of 
collaboration or computer-supported work in synchronous 
or asynchronous mode, and potentially the technical prob- 
lems are further compounded by the opportunity for differ- 
entiated and value-added services increases, i.e., the double- 45 
edged sword once again. 

The best way to understand a system is to have an 
abstraction that describes a simpler picture of the structure 
and the machinery. A metaphoric vehicle is useful in that it 
allows framing of a problem and likewise offers a solution 50 
that supports and promotes flexibility, expressiveness, and 
scalability in information design and display. One can say 
that a multi-media presentation is like "telling a story". The 
presentation author is attempting to convey a communica- 
tive intent and more than likely it was constructed with a 55 
particular audience in mind, as well as a specific context and 
medium. 

The computational narrative model, as disclosed in 
Brooks, K. M., "Do Agent Stories Use Rocking Chairs: The 
Theory and Implementation of One Model for Computa- 60 
tional Narrative", Processings of the Fourth ACM Interna- 
tional Multimedia Conference on Intelligent User Interfaces, 
ACM Press 1996 and Murtaugh, M. "The Automatist Sto- 
rytelling System: Putting the Editor's Knowledge in 
Software", MIT MS Thesis, 1996, offers a metaphor for 65 
creating tools that are capable of going beyond traditional 
storytelling by enhancing the editorial through the leverag- 



ing of the computer's ability to support rapid decision 
making. According to Brooks, narrative represents the uni- 
verse of story elements for a given story, i.e., the collection 
of possibility, and narration as a specific navigation through 
that universe. 

As shown in FIG. 1, the process of computational story- 
telling involves the author supplying the elements of the 
story and the structure to organize the story elements. The 
agent takes the elements of the story and the structure and 
generates a story, more precisely, a narrative, and presents 
the "story" to an audience. The audience reacts and gener- 
ates feedback to the agent. The agent acting as proxy for the 
author can react to the feedback by modifying the presen- 
tation. 

Some current conceptual views regarding the techniques 
or technical strategies that are related to developing a 
framework for creating and delivering interactive multime- 
dia applications include: dynamic presentation, behavior- 
based artificial intelligence, memory-based learning, and 
user modeling. 

Regarding dynamic presentation, Maybury, M. "Intelli- 
gent Multimedia Interfaces", AAAI/MIT Press, Cambridge, 
Mass., 1993, discloses that automatic multimedia presenta- 
tion involves the stages of content selection (i.e., what to 
say), media allocation (i.e., what media to present it in), and 
media realization (i.e., how to say it). The focus is the media 
allocation and realization phase. More specifically, how to 
create presentations without knowing all "facts" during 
design time. The basic objective is to enable the creation of 
user interfaces that are sufficiently flexible and adaptive to 
"re-invent" themselves at run-time. To support this flexibil- 
ity and adaptability, an interface needs to be developed not 
to a final fixed form, but to some protean form that can be 
reshaped at run time, time after time, to meet the require- 
ments of any situation that invalidates its current form. 

Szekely P., "Retrospective and Challenges for Model- 
Based Interface Development", USC Information Sciences 
Institute, Marina del Rey, Calif., 1996, proposes one archi- 
tecture. Szekely discloses that a model-based user interface 
calls for a model of the interface that is organized as three 
levels of abstraction: task and domain model for the 
application, an abstract user interface specification, and a 
concrete user interface specification. The task model repre- 
sents the task that the user will undertake to perform with the 
application. The domain model represents the data and the 
operations that are part of an application. 

The second level, according to Szekely, is the abstract 
user interface specification. At this level, an interface is 
defined in terms of abstract interaction units, information 
elements, and presentation units. The abstract interaction 
units are low-level interactions such as showing a presen- 
tation unit. Information elements represent data such as 
attributes extracted from the domain model. Presentation 
units are abstractions of windows and specify collections of 
abstract presentation units and information elements that are 
to be treated as a unit. Basically, the abstract user interface 
specification abstractly specifies the way information will be 
presented in the interface and form for interaction with the 
information. 

The third level, according to Szekely, is the concrete user 
interface specification that specifies rendering styles for the 
presentation units, i.e., widgets. Different model-based user 
interface (UI) frameworks differ in what models they pro- 
vide. Szekely discloses that some frameworks have one 
model but not the other two, while in other cases, only one 
model is defined. FIG. 2 is a flowchart showing a generic 
model-based presentation system as disclosed in Szekely. 
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An alternative reasoning framework has emerged in Arti- 
ficial Intelligence circles called Behavior- Based AI (BBAI) 
as disclosed in Maes, P. "Behavior-Based Artificial 
Intelligence", Proceedings of Second Animat Conference on 
Adaptive Behavior, 1992. This new approach represents 5 
more of a different way of thinking about a problem domain 
than an alternative reasoning technique. The knowledge- 
based approach involves capturing the rules to solve a 
domain. In contrast, the BBAI approach relies on a set of 
lower level competencies which are each experts at solving 10 
one part of the larger problem domain as disclosed in 
Brooks. 

Additionally, the BBAI approach tends to emphasize the 
system behavior as opposed to the system knowledge. 
Furthermore, BBAI stresses that the system should be situ- 15 
ated in its environment and have direct (or as close as 
possible) access to the problem domain. This framework 
enables a system to bring together different classes of 
reasoning techniques, heuristic, statistical, etc., and incor- 
porate each application of a technique into a lower-level 20 
competency module or "expert". In effect, these modules 
come together to form a multi-agent system. 

Another learning technique, as disclosed in Stanfield, C. 
et al., "Toward Memory -Based Reasoning", Communica- 
tions of the ACM, 20(12), ACM Press, 1986, is memory- 25 
based learning. Basically, memory-based learning entails 
comparing a new situation against each of the situations 
which have occurred before. Given a new situation, a 
memory-based learning agent looks at the actions taken in N 
of the "closest" situations or "nearest neighbors" to predict 30 
the action for a new situation. FIG. 3 shows a diagram of the 
memory-based reasoning approach. 

User modeling is an inexact science but its predictions 
need not be perfect to be useful. User models can range from 35 
simply storing a bit indicating if the user is a novice or expert 
in terms of an application, to a rich, complex snapshot of the 
user's interest and preferences. Once a universe of user 
models is collected and maintained, the models may serve as 
data for further analysis to find pattern and trends in this ^ 
universe. These are some of many critical issues relevant to 
user modeling. 

User models may be either pragmatic or cognitive as 
disclosed in Orwant, J, "Doppelganger Goes To School: 
Machine Learning for User Modeling,", MIT MS Thesis, 45 
1993. The cognitive type user models are not connected to 
any application or applications in particular. This type of 
user model is attempting to capture a user's beliefs, goals 
and plans in a general sense. A pragmatic user model is not 
driven by a cognitive model but by the practical aspects of 50 
the environment, e.g., applications. The pragmatic user 
model can be characterized by the collection of raw obser- 
vational data and making sense of the data after the fact. In 
another sense, the cognitive model is a top down approach 
and the pragmatic model is a bottom up approach. S5 

Conceptually, individuals can take on particular roles, 
e.g., business, leisure, parental, professional. These are 
defined as persona in a user modeling sense. Personae could 
be utilized to partition the user model space into more 
manageable chunks. so 

A pragmatic user model can make use of filtering tech- 
niques. Content-based filtering involves selecting items for 
the user based on correlations between content of the items 
and the user's preferences. For example, a personalized TV 
program guide uses information about a television program, 65 
such as the program's type and its level of violence to 
predict whether or not to recommend and include the show 
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in a personalized line-up. Generally, users rely on explora- 
tion to discover new items of interest, i.e., serendipitous 
items. By definition, content-based filtering has no inherent 
capability to generate these sort of items. In practice, one 
must add special purpose techniques to add these capabili- 
ties to content-based filtering to introduce serendipity. For 
example, a user might be unaware of their interest in true 
crime shows until she actually comes across "America's 
Most Wanted". Assuming no indications of this trend had 
previously surfaced, content-based filtering would have 
never detected this particular interest. Content-based filter- 
ing simply does not allow a user to expand their interests. 

Social-based filtering is one potential solution to the 
serendipity dilemma. Social-based filtefing^j>asically 
attempts to exploit similarities between the profiles of dif- 
ferent users to filter content .Social-based filtering can be an 
extension of content-bas^^fiering. Once a user model is 
constructed and is beingTrnairitained, social-based filtering 
algorithms can compafe^this model to other user models and 
weigh each model fo^^e level of similarity with the user 
model. Orwant, J., "For Want of a Bit The User Was Lost: 
Cheap User Modeling", IBM Systems Journal, vol. 35, Nos 
3&4, 1996 and Shardanand, U., "Social Information Filter- 
ing for Music Recommendation", MIT MS Thesis, 1994 
disclose algorithms for computing similarity between user 
models. 

SUMMARY OF THE INVENTION 

Accordingly, the present invention is directed to a method 
for design of an adaptive personalized interactive content 
delivery system that substantially obviates one or more of 
the problems arising from the limitations and disadvantages 
of the related art. 

It is an object of the present invention to provide an 
architectural framework that is composed of a collection of 
classes for building interactive multimedia applications and 
services. 

It is a further object of the present invention to provide an 
architectural framework that will enable a developer to build 
up locations that deliver services that dynamically adapt to 
the user, the content, and the delivery context, resulting in an 
effective contextual personalized on-line experience. 

Another object of the present invention is to provide an 
architectural framework that supports and promotes the 
creation of reusable components for building personalized 
interactive multimedia presentations of complex applica- 
tions. 

Accordingly, one aspect of the present invention is 
directed to a method for creating and delivering an interac- 
tive multmieifia^pplication that can dynamicallyiadapf to at 
least one user. At least one user model is created for at least 
one userTJEeat least one user model represents interests and 
trends of the at least one user. A multimedia story is 
developed based on the at least one user modeirAcustom- 
ized presentation of the multimedia story is^nerated where 
the at least w o^rmultimedia story allows for multiple pre- 
sentations of the multimeoUastory. The customized presen- 
tation is displayed to the aFleast one user. The customized 
presentation is modified based on input from the at least one 
user. 

In another aspect of the present invention, the story 
includes a protean-like narrative. 

In still another aspect of the present invention, the creat- 
ing includes: gathering data from the at least one user; 
analyzing a history of the at least one user; monitoring data 
related to the at least one user; detecting patterns and trends 
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of the at Least one user; and preparing the at least one user presentation model, the abstract presentation may be gen- 
model based on the gathering, analyzing, monitoring, and erated by the presentation engine; generating a concrete 
detecting. The at least one user model is modified periodi- presentation by using the abstract presentation's heuristics, 
cally based on information obtained from periodically the concrete presentation may be generated by the presen- 
repeating the gathering, analyzing, monitoring, and detect- 5 tation engine; and displaying the concrete presentation by 
ing. the presentation engine, wherein the abstract presentation 

In a further aspect of the present invention, the at least one ^ mc presentation engine autonomously handle interac- 

user model includes a set of models. ^on scenarios, and trends and patterns are periodically 

In another aspect of the present invention, the story recomputed based on interaction histories and the user 
includes at least one content element. The at least one 10 modcls : ^ interactive multimedia application may be self- 
content element characterizes data of the interactive multi- improving and self-sustaining. 

media application. The at least one content element is to another aspect of the present invention, the interactive 

representable in multiple forms. multimedia application may be created using object-oriented 

In still another aspect of the present invention, the at least design techniques, 
one user model comprises a set of models. In still another aspect of the present invention, the inter- 
In a further aspect of the present invention, the multiple activc ™ ltime ^ application may be created using JAVA, 
forms include text, audio, video, image, or multimedia./ In a further aspect of the present invention, the invention 
In another aspect of the present mvention,lh7S^tion * t0 a ^ stem and delivering interactive 
includes filtering the at least one content element to produce 20 multmiedia apphcaUons that dynamically adapt to auser that 
a subset of the at least one content element, each content mclude: a user modelmg subsystem where the user modeling 
element in the subset of at least one content elements subsystem creates and maintams at least one user model for 
selected based on semantics of the filtering. ™°} ^ e * ch a ' leasl one ^ model presents interests 

~ . , . and trends of each user; a story engine subsystem where the 
In still another aspect of ^the present ^invention, the inven- st0 ry engine subsystem selects appropriate content elements 
Hon includes assembling the subset of at least one content 25 ^ ^ ^ ^ elements ^ accordance 
elements to produce the multimedia story. The multimedia ^ a narraUye ^ a presentation subsystem 
story may be personalized to the at least one user. where the presentation subsystem generates a presentation to 
In a further aspect of the present invention, the generating the user, the presentation generated uses the narrative frame- 
includes: determining the delivery environment of the at work. 

least one user; determining the style look and feel for the 30 In '^tv aspect of the present invention, the user mod- 
presentation; determining the narrative context for the pre- eling subsys t em includes: a user model editor; a user mod- 
ulation. The narrative context defined by the semantics of eling manager; an m ^ ysis engine; ^ a ^ model data . 
the interactive multimedia application; and creating a cus- base. 

tomized presentation of the multimedia story based on the T *-n *u * * a. *• *• 4 l * 

, v . > ,i 4 i i , , r i • 35 to still another aspect of the present invention, the story 

delivery environment, the style look and feel, and the « . • i j c * j * i_ u lL £ , 

narrative context engine subsystem includes: a first database where the first 

database contains a content model library, the first database 

In another aspect of the present invention, a weighted accesses content from a content database; and a second 

value may be assigned to each interest and trend of the at database where the second database contains a story tem- 

least one user. The weighted value represents the relative ^ pj^g library 

importance of each interest and trend with respect to the at ^ a ^ of ^ { ^ ^ 

least one user s apparent mterests. don subsystem indndfiK a first database where F me firsl 

In still another aspect of the present invention, the inter- database contains at least one presentation models; a pre- 
active multimedia application may be created using object- sentation builder; a second database where the second 
oriented design techniques. 45 da tabase contains a concrete presentation library; and a 

In a further aspect of the present invention, the invention presentation engine, 

is directed to a method for creating and delivering an i n another aspect of the present invention, the content 

interactive multimedia application that can dynamically elements may represent pieces of information that can be 

adapt to at least one user that includes: creating a story presented via one or more media types, 

engine, the story engine may be created by the interactive 50 In still ^ ^ cci of ^ ^ inven tion, the pre- 

multunedia application; creating a user model manager, the sentation be constrained by a narrative style narrative 

user model manager may be created by the interactive context> md demands of me deli environment of the 

multimedia application; providing the story engine with user 

application-specific information and user information; pro- ~" . , . . . . 

■ . -*u a i c ~ Other exemplary embodiments and advantages of the 

viding the story engme with a user model from the user 55 . . J * ^ - , t . - 

model managed, the user model represents interests and invention may be ascertained by reviewing the 

trends of the at least one user; providing the story engine present and the accompanying drawings, 

with a narrative structure, the narrative structure defined by BRIEF DESCRIPTION OF THE DRAWINGS 

the semantics of the interactive multimedia application; „_ 

producing user-relevant content, the user-related content 60 FIG - } 15 a flowchart showing a conventional dynamic 

may be produced by applying filters to the content model, storytelling structure; 

the user model may be used for filtering purposes; creating FIG * 2 is a flowchart showing a conventional generic 

a presentation engine, the presentation engine may be ere- model-based presentation system; 

ated by the interactive multimedia application; providing the FIG. 3 is a diagram showing a conventional memory- 
presentation engine with the narrative structure, content 65 based reasoning system; 

model, and a presentation model, the content model may be FIG. 4 is a flowchart showing a multiagent storytelling 

empty; generating an abstract presentation defined by the system according to the present invention; 
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FIG. 5 is a diagram showing a model abstraction view the present invention only and are presented and the cause 

controller architecture according to the present invention; of providing a useful and readily understood description of 

FIG. 6 is a flow diagram showing a functional overview the principles and conceptual aspects of the present inven- 

of an application's framework according to the present tion. In this regard, no attempt is made to show structural 

invention; 5 details of the present invention in more detail than is 

FIG. 7 is a flow diagram showing an architectural frame- necessary for the fundamental understanding of the present 

work system architecture according to the present invention; invention. The description taken with the drawings make it 

FIG. 8 is a flowchart showing an exemplary presentation apparent to those skilled in the art how the several forms of 

object model according to the present invention; the present invention may be embodied in practice. 

FIG. 9 is a flowchart showing an exemplary object model io The present invention is an application framework for 

for framework according to the present invention; creating and delivering interactive multimedia applications 

FIG. 10 is a flow diagram of a model of content and story and/or services. The applications framework according to 

and an exemplary representative application according to the the present invention will enable the deployment of appli- 

present invention; cations that dynamically adapt to the user through the 

FIG. 11 is a flowchart of an exemplary object model for 15 personaUzation^of content and'^r^mation^Tbe-applica- 

a representative application according to the present inven- Uon3mfih'ework may be^a sof tw£re mfr that sup- 

tion; ports and promotes the^eatidlr^fireusa^ for 

FIG. 12 is an exemplary interaction diagram for boot- building personaUzej|jnteractive multimedia^presentations 

strapping use case according to the present invention; of complex applications. In addition, the applications frame- 

FIG. 13 is a flowchart showing the relationships between 2 o WOf k according to the present invention, is a software 

a community model, user models, and user personae accord- foundation for enabling community and collaboration in a 

ing to the present invention; networked world. 

FIG. 14 is a diagram showing semantics and content; The applications framework, according to the present 

FIG. 15 is a flow diagram showing multiple representa- invention, allows one to create an appUcation-specific struc- 

tions of content according to the present invention; 25 ture and utilizes structure to create multiple presentations 

FIG. 16 is a diagram showing selective assembly of from the same set of application-specific content where 

content according to the present invention; agents with different style goals or communicative intent 

FIG. 17 is a diagram showing an anatomy of an applica- make sequencing and editing decisions constrained by the 

tion according to the present invention; user's preferences and the characteristics of the content and 

FIG. 18 is a diagram showing a thick client-thin server 30 the delivery device, 

partitioning of an application according to the present inven- As discussed previously, the best way to understand a 

tion; system is to have an abstraction that describes a simpler 

FIG. 19 is a diagram showing a thin client-thick server picture of the structure and the machinery. The architecture 

partitioning of an application according to the present inven- of an applications framework according to the present 

tion; 35 invention may be described by a series of abstractions, each 

FIG. 20 is a diagram showing a peer-to-peer distributed one giving more and more concrete artifacts. The application 

partitioning of an application according to the present inven- framework according to the present invention encompasses 

tion; many elements ranging from a dynamic presentation system, 

FIG. 21 is a user modeling class diagram according to the a multiagent system, to a memory-based user modeling 

present invention; 40 system and a multi-paradigm application framework. 

FIG. 22 (consisting of FIGS. 22-1 and 22-2) is a story Reflecting on the originally discussed metaphor, the 

engine class diagram according to the present invention; present invention decomposes the agent that inhabits the 

FIG. 23 (consisting of FIGS. 23-1 and 23-2) is a presen- dynamic storytelling structure down to a set of agents, each 

tation engine class diagram according to the present inven- a S cnt corresponding to an area of competency. These 

ti on; 45 mclud^ser^pjding, storytelling, and presentation design/ 

FIG. 24 is a content classes class diagram according to the gcnerationr This model subscribes to the behavio^based AI 

present invention- approach where^eaeb agent, an expert in its own nght;brings 

FIG. 25 is a metadata classes class diagram according to 0WI \ lower J?£5 ^ u £f tencics to crcatc a 

the present invention; <n higher level competence -emergent behavior. 

FIG. 26 is a block diagram of exemplary content database . d # 1SCUSSed P revious1 ^ the requirement for media 

according to the present invention; ^1 * * ^^j™ of in °™ atlOD 

n„ *- . 11 1 j. c i_ * i_ 1 1 • r where the dynamics are characterized by: constantly chang- 

FIG 27 is a block diagram of a high level view of an ^ fofonwlioii, broad user population, and heterogeneous 

eXC ^? 7o W T f ^ t landscape of delivery devices. The architectural framework, 

FIG. 28 is a flowchart of an exemplary story model 55 acc0 rding to the present invention, solves the challenge of 

according to the present invention; constantly changing information with the story agent, the 

FIG. 29 is a flowchart of exemplary HTML presentation challenge of broad user population with the user agent, and 

templates according to the present invention; tne challenge of heterogeneous landscape of delivery 

FIG. 30 is a flowchart of generation of a presentation devices with the presentation agent. FIG. 4 shows a diagram 

structure according to the present invention; and 60 0 f a mu ltiagent storytelling system according to the present 

FIG. 31 is a flowchart of an exemplary final form of a invention. The user agent, story agent and presentation 

presentation of a scene according to the present invention. agent, according to the present invention, will now be 



DETAILED DESCRIPTION OF THE 



discussed in more detail. 



PREFERRED EMBODIMENTS 65 User^g|iit 

The particulars shown herein are by way of example and The User Agent embodies the user modeling aspect of the 
for purposes of illustrative discussion of the embodiments of architectural framework system, according to the present 
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invention. In effect, the user modeling system is the user to 
the system. It encompasses several components that together 
enable the capture of relevant interaction data, the storage 
and representation of a user's interests, trends, etc., and the 
capability to manage and analyze the resulting user data. The 5 
User Agent in the architectural framework, according to the 
present invention, handles capturing user feedback, main- 
taining the user's profile, structuring interests and 
preferences, and making sense of a user's interaction history. 
A User Model Editor allows the end user and/or adminis- 10 
trator to specify the user's interest along with a measure of 
confidence. 

A sensor is used to capture user interaction at the source 
and understands how to extract the relevant information 
from the user feedback. The sensor may be in the form of a 15 
sofrware^rdgram. The sensor acts as a proxy for the user 
modelingnsystem 1 . Different kinds of sensors may be 
employed to gather information at their respective sources. 
A sensor knows how often to gather data, what data to 
monitor, and how to decode the present event into user 20 
profile data. A sensor may be one or several software 
components, where each component may capture and/or 
monitor different user information. 

The user modeling system, according to the present 
invention, provides for a repository for representations of 25 
each users' preferences. A user's preference and taste, along 
with demographic information, constitutes a user model. 
Additionally, each user model needs to maintain some form 
of history that describes the relevant "discourse" of inter- 
action that supports the user's preferences contained therein. 30 
Sensors provide the interaction data. 

In the architectural framework, according to the present 
invention, the nature of the representation of a user model is 
driven by the feature-based content that characterize the 35 
application data. As a result, the user models are structured 
as a set of models for each domain or application (e.g., TV 
viewing, shopping, etc.). This is in contrast to the persona 
concept described previously. Persona relate to a role rather 
than an application's specific profile. A persona is a model ^ 
that exists independent of an application oriented or a 
domain oriented model. 

In order for the user models to be useful to other com- 
ponents in the architectural framework of the present inven- 
tion (e.g., the Story Agent), the User Agent is introspective 45 
and computes/detects trends and patterns. The User Agent 
constantly re-evaluates the importance of features and the 
values the features can hold in the domain oriented models. 
In the architectural framework according to the present 
invention, the User Agent includes a reasoning component, 50 
an analysis engine, that analyzes a user's data and computes 
correlations between features and feature-values as defined 
by the memory based learning framework described previ- 
ously. 

Story Agent 55 

The Story Agent according to the architectural framework 
of the present invention, selects the appropriate content 
elements and collects and organizes these elements as pre- 
scribed by an appropriate class of narrative framework. This so 
narrative framework represents a "prototype story" that is 
utilized by the Presentation Agent to generate a customized 
presentation. 

The process of selecting content is driven by the content 
types as specified by the content model. The User Agent's 65 
user model is utilized in the selection process. As described 
previously, the User Agent is responsible for analyzing a 
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user's data and computing correlations between features and 
feature-values and carrying out its role in the memory based 
learning scheme. 

Given some application-specific criteria, the Story Agent 
is responsible for choosing the best content elements by 
using the embedded logic provided by the narrative frame- 
work. Once the story agent has selected the "best set" of 
content elements, the narrative framework now populated 
with content elements is supplied to the Presentation Agent's 
computation of the final story's look and feel. 

Presentation Agent 

Using the dynamic storytelling metaphor discussed 
previously, the Presentation Agent according to the present 
invention takes the dynamically generated narrative (the 
populated narrative framework) and creates a presentation. 
The Presentation Agent generates a presentation by design 
where the design is constrained by a narrative style 
(narrative context), by a particular look and feel (style 
context), and by the demands of the delivery environment 
(delivery context). The agency of this agent is brought to life 
by specifying an abstract representation of a presentation. 
The model based approach to user interface design supports 
the idea of an abstract, declarative representation of a user 
interface. The approach according to the present invention, 
borrows from this approach, but only superficially, mainly in 
high level terms. 

The presentation generation aspect of the architectural 
framework according to the present invention is a novel yet 
simple solution. Presentation design involves four types of 
components: abstract presentations, concrete presentations, 
reactors, and design constraints. 

An abstract presentation is a meta-presentation. An 
abstract presentation is a loose design representation of a 
concrete presentation. Abstract presentations may have parts 
that themselves are abstract presentations. This results in the 
creation of an hierarchically composed presentation. Defin- 
ing a modular loosely structured presentation not restricted 
to a final form and layout enables the creation and mainte- 
nance of flexible and dynamic multimedia presentations. An 
abstract presentation maps to and represents a content ele- 
ment. It serves as the link between interface and content 

Concrete presentations are either standard user interface 
components, Le., widgets or wrapper-style components that 
repurpose other widgets. A wrapper is a software component 
(e.g. an object) that hides the real software or device that is 
being used. The concrete presentation objects are the actual 
user interface objects that appear on the display. 

Reactors are action objects that associate concrete pre- 
sentation events and an operation on a content element. 
Reactors are registered (i.e., associated with appropriate 
software components to handle the operation) and managed 
by abstract presentations. 

Design constraints are heuristics that guide in the final 
make-up of the presentation, including layout, style and 
content make-up of the presentation. These rules can be 
classified into three categories: narrative context, style 
context, and delivery context. Narrative context are rules for 
narrative-specific realization, e.g., in a personalized TV 
program guide creating programs, line-ups along thematic 
lines. Style context are rules for style, look and feel, e.g., a 
tabular view versus a 3D landscape of the schedule in a 
personalized TV program schedule. Delivery context are 
rules that deal with the delivery environment, e.g., real estate 
allocated for a desk top versus a PDA (Personal Digital 
Assistant), connection protocol, browser, modem^jeed^ etc. 
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Hie abstract presentation and design constraints represent sentation model, and a domain user model for the user 

the declarative aspects of the architectural framework model. Creating a content model is building a typical model 

according to the present invention, and together they serve of the application, such as called for in the traditional 

as an abstraction of the final interface. As shown in FIG. 5, Mo del- View-Controller sense. The content model is a rep- 

this framework is an extension of the known ModeUView- 5 resentation of the semantics that characterize the content 

Controllers(MVC)^user irUerface^architecture. The MVC elements that make up an application, e.g., a TV program 

paradigm partitions a user mterface T arch^ctiirc; into three schedule consisting of program line-ups where the line-ups 
components: a £ moclel~(an^:abstraction^ r of Z IheTproblem ^ consist of time slots populated by TV programs slated to be 
domain), a view (a^visual rcpresentation^ofthe model or partly broadcast. 

of the model), and :a^ntfoller-(3^put handler). Typically^So The Story Model (narrative structure) is a "protean"-like 

each of these components is a collection of one or more content model. It serves to organize the content elements 

objects. mat oave b een se i ec ted as candidates for the presentation 

In the known MVC user interface, architecture, the user generation phase. The narrative is basically the universe of 

issues some form of input or command which is captured by possibilities as defined by the semantics of the application, 

the controller. The controller, in turn, takes the command 15 e.g., creating a personalize^gfcf .program guide that presents 

(e.g., mouse click, key stroke, speech utterance, etc.) and a set of personalized line-ups involves a narrative structure 

translates it into an action on a model object. A controller is that groups candidate programs according to their start time 

associated with a view. The view displays the state of the (candidacy is a complex step of consulting a user model and 

model and relies on a dependency mechanism whereby the predicting the best content elements given a set of applica- 

view registers itself as dependent on the model. As a result, 20 ti on specific criteria). 

whenever the model undergoes a change, the model broad- ^ presentation model specifies the components of a 

casts a notification to all dependent views that a change has presentation, the behavior (Unking actions in the presenta- 

occurred. The views in turn query the model to retrieve the tion ^ to application content functions) and design heu- 

details of the change and update themselves accordingly. ristics (rules mat ^ Ktdng ^ present ation style, pre- 

The model encapsulates the semantics of application sentation context, and display context), e.g., in the 

objects. Subcomponents of the model hide the details of personalized TV program guide, if the presentation context 

communication, database access, etc. The model is not is thematic then generate a personalized line-up where each 

aware of the views (or the controllers for that matter) but line-up represents a particular theme given the candidate set. 

only through anonymous message broadcasting does the The Presentation Builder stores the presentation model in 

model communicate with its dependent views. 30 persistent storage. 

The MVC architecture promotes modularity and the Building a domain model for the user model involves 

decoupling of application data from the mechanisms to view accounting for the features that make up the content ele- 

and manipulate that data. So as a result, this allows for ments in an application, e.g., using the personalized TV~ 

software reuse, bom in a design and implementation sense. 35 program^guide once again, features would include, e.g., 

In theory, one can reuse a model in different application, i.e., program^ type, level of violence, etc. 
the same model, different views. Additionally, one may 

reuse a view (or controller) in different applications, i.e., Architecture/Subsystems 
same view, different models. 

The known MVC architecture assumes a set of views 40 FIG ' 7 shows a flowchart of «* exemplary system archi- 
statically bound to each model. The architectural framework tecture accordm g to ™ architectural framework according to 
according to the present invention, has extended this archi- me P resent mve ^on. The User Model Manager manages 
tecture by decomposing the view controller compliment set me stora S e and retneval of user ' s interests and trends and 
into an abstract presentation component and a concrete interfaces to other subsystems. The Analysis Engine is used 
presentation component as shown in FIG. 5. The concrete 45 l ° T ^ Iac ^ ^tories detect P atterns 
component encapsulates the traditional view-controller trends. The User Model Eo^tor is an admimstraUve tool mat 
objects, but only in an incomplete and unrealized state. The ^ ows a administrator to modify a user model, 
abstract component dynamically generates and manages the ^ user modeling subsystem uses user models and corn- 
final form look and feel of the concrete components. munity models. A community model is a model of a group 
Moreover, by representing the interface in abstract terms, the 50 of 05618 mat share *° mQ common mterest or trend " 
present invention effectively enables the creation of dynami- The Story Engine selects application-specific content to 
cally bound views not possible in the currently known MVC serve as the addition for a new presentation, and generates 
tradition. The architectural framework according to the a narrative/story that alio ws for multiple play out of different 
present invention, defines a declarative based Model- presentations of the story. The Story Engine uses the story 
Abstraction- View-Controller user interface architecture. 55 model and the user model. The Presentation Engine is 

FIG. 6 shows a functional overview of an application responsible for interpreting an abstract presentation model 
framework according to the present invention. In FIG. 6, the and creating concrete presentation objects. The Presentation 
items in the circles represent subsystems. The items inside Engine also resolves constraints imposed by abstract 
the parallel lines generally represent models, except for the presentations, input content, and display context as part of 
user feedback. The process of developing an application 60 the final realization of the concrete presentation objects. The 
using an application framework according to the present Presentation Engine uses the presentation model and the 
invention, consists of designing or specifying (and possibly storv model - The Presentation Builder is responsible for 
reusing or repurposing pre-existing models) models required storing presentation models in persistent form. The Presen- 
by the agents of the framework. Basically, the models for tation Builder uses the presentation model, 
content presentation and the user need to be created. The 65 Designing and delivering an application using an archi- 
designer of the application must design and specify: a tectural framework according to the present invention gen- 
content model, a story model (narrative structure), a pre- erally include: 
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1. The application creates a Story Engine (SE) and a User 
Model Manager (UMM). 

2. The application informs the SE who the user is and any 
other application-specific information deemed neces- 
sary. 

3. The SE requests a user model from the UMM. 

4. Upon receiving a user model from the UMM, the SE is 
handed a narrative structure as defined by the applica- 
tion semantics. 

5. Applying filters contained in the narrative framework, 
the SE places the results (i.e., content elements) in the 
narrative structure. 

6. The application creates a Presentation Engine (PE). 

7. The PE is handed the narrative structure for the 
application, the content model, and the appropriate 
presentation model. 

8. The PE generates an abstract presentation as defined by 
a presentation model. 

9. The PE exercises the abstract presentation's heuristics 
and generates a concrete presentation. 

10. The PE displays the concrete presentation. 
The application is self-sustaining at this point. The 

abstract presentation, along with the presentation engine, 
handle autonomously most interaction scenarios using the 
flexible and adaptable capabilities encapsulated in the pre- 
sentation. Two basic scenarios exist that will violate this 
state. First, the user requests for content data that did not 
play a role in the story generation (e.g., in the personalized 
TV program guide, personalized line-ups from 6:00 p.m. to 
9:00 p.m. are presented, but the user now wants to expand 
the window of the program guide by looking at programs 
from 6:00 p.m. to 12:00 a.m.). In the second scenario, there 
is a change to the content model and its elements, requiring 
generation of the story by re-evaluating the narrative and 35 
recreating the presentation (e.g., in the personalized TV 
program guide, a programming change has occurred and a 
new show has been scheduled). Both of these scenarios 
involve executing steps 5-8. 

At appropriate times (e.g., overnight), the Analysis 40 
Engine examines the interaction histories and the user 
models and recomputes trends and patterns. The user models 
are then revised accordingly. An architectural framework 
system according to the present invention is thus self- 
improving, self-sustaining and virtually perpetual. 45 

Abstract Class Specifications 

Some exemplary object models for the architectural 
framework according to the present invention follow. FIG. 8 50 
shows an exemplary presentation object model according to 
the present invention. FIG, 9 shows an exemplary object 
model for the architectural framework according to the 
present invention. The various boxes in the object models 
represent classes of objects of the architectural framework. 55 
The following tables list the classes along with their asso- 
ciated responsibilities and attributes. 



10 



15 



20 



25 



30 



Responsibilities 



Presentation Classes 
Attributes 
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-continued 



Responsibilities 



Presentation Classes 



Attributes 



Manage a set of subordinate 
presentations 
Add a presentation 
Delete a presentation 
ConcretePresentation 

Interface to windowing/GUI 

environment 

Sensor 



a ConcretePresentation 

set of associated constraints (rules) 



a ConcretePresentation 



Reports user behavior to UserModel- 
Manager 

Monitors for specific events 
Reactor 

Encapsulates an application-specific 
behavior 

Acts as an action/command object 
PresentationEngine 

Creates and displays 
Abstract Presentations 
Interprets the declarative specification 
associated with an Abstract- 
Presentation 

Reports invalidated presentations to 
Application 

Resolves presentation's constraints 
and realizes CoacretePresentations 
Presents tionBuildej 



an AbstractPresentation 

an Event Type 

a Us erMo del Manager 



an AbstractPresentation 
a ContenLElemcnl 



an AbstractPresentation (top-level) 
an Application 



Stores AbstractPresentation in 
persistent storage (e.g., file) 



an AbstractPresentation (top-level) 



Content Classes 



ContentElement 



Represents an application-specific 
object 

Story Classes 



StoryEngine 

Selects ContentElements as specified 
by Story type and filtered by the 
UserModel 

Creates a Story structure 
Story 



Represents a particular narrative 
structure, application-specific 

User Modeling Classes 



a UserModel 

a ContentElement (top-level) 
a Story 



set of ContentElements 



UserModel 



60 



Maintain multiple personae 
Add a persona 
Remove a persona 
Find a persona 
Persona 

Add a situation-action pair 
Remove situation-action pair 

Find situation-action pair 
Community 



AbstractPresentation 

Serve as prototype for a Concrete- set of presentations 
Presentation set of reactors 



Add UserModel 
Delete UserModel 
65 Construct Average UserModel 
Find UserModel 



set of personal data 
set of Persona 



set of preferences 

set of situation-action pairs 

(history) 



set of UserModels 
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-continued 
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Responsibilities 



Presentation Qasses 



Attributes 



Society 

Add Community 
Delete Community 
Construct Average Community 
Find Community 
UscrModelManagcr 

Gateway to AnalysisEngine, 
UserModelEditor, and UserModels 
Requests for Sensor from 
PresentationEngine 
AnalysisEngine 



Performs historical/trend analysis on a UserModel 

UserModel's histories 

UserModelEditor 



set of Communities 



set of Models 



Presents a UseiModel's set of 
persona 

Presents a persona 



a UserModel 



Application Classes 



Application 

Sequences the "tools" to create a 
presentation 

Handle application-specific events 
(e.g., invalidated presentations, 
special timers) 



10 



15 



20 



25 



a PresentationEngine 
a UserModelManager 
a StoryEngine 



35 



40 



An exemplary representative application will be defined 
and used to illustrate the capabilities of the architectural 
framework according to the present invention. This repre- 
sentative application is a TV program guide. The exemplary 
TV program guide is a personalized program guide (PPG) 
that suggests TV programs that may be of interest to the user 
right along side the traditional program schedule. The fol- 
lowing assumptions will be used: (1) the presentation model 
has been previously specified and declared; (2) the applica- 
tion is always up and running (i.e., 24 hrs a day); and (3) the 
Analysis Engine has conducted its initial analysis of the 
viewer's history. 

A content model is defined in order to create the appli- 45 
cation. FIG. 10 shows an exemplary object model of content 
and story in the exemplary application. RG. 11 shows a 
flowchart for an exemplary object model for the exemplary 
application. In FIG. 9, the run time representation of the 
overall exemplary application is outlined. The PPG appli- 
cation displays a program guide that includes three areas: the 
current movie playback component, a current informational 
panel, and the program schedule grid. 

An exemplary case that demonstrates the mechanics and 
structure of the architectural framework according to the 55 
present invention will now be presented. This exemplary 
case relates to bootstrapping an application from its initial 
interaction with the user modeling system and the story 
construction process, to the initial presentation and event 
handling by the presentation engine. Two assumptions have 60 
been made: (1) the presentation model has been previously 
specified and declared; and (2) the Analysis Engine has 
conducted its initial analysis of the viewer's history. FIG. 12 
shows an exemplary interaction diagram for this exemplary 
bootstrapping use case according to the architectural frame- 
work of the present invention. The following activities occur 
during this bootstrapping: 



50 



65 



(1) anApplication creates a User Model Manager 
(aUserModelMgr); 

(2) anApplication creates a Story Engine (aStoryEngine); 

(3) anapplication creates the standard Program Schedule 
(aProgramSchedule) based on some initial time bound- 
aries; 

(4) aStoryEngine requests a user model based on a 
Name/ID from the User Model Manager 
(aUserModelMgr); 

(5) aStoryEngine retrieves the program schedule 
(aProgramSchedule); 

(6) aStoryEngine selects appropriate application content 
based on the user model (atJserModel) and the input 
content (aProgramSchedule); 

(7) aStoryEngine generates a story based on a story 
template program guide narrative 
(aPgmGuideNarrative); 

(8) anApplication creates an instant of a Presentation 
Engine (aPresentationEngine); 

(9) aPresentationEngine creates an abstract presentation 
(likely a series of nested presentations^ by restoring the 
object from persistent story, e.g., straining from a file; 

(10) aPresentationEngine creates all specified interactors 
for each abstract presentation. In this example, aSe- 
lectCmd interactor. 

(11) aPresentationEngine creates a grid object to aid in the 
layout of the overall presentation; 

(12) aPresentationEngine creates all concrete presentation 
objects as declared by their corresponding abstract 
presentation; 

(13) aPresentationEngine resolves constraints as specified 
by the display rules and application rules and recon- 
ciled with the input content and the display context by 
the aPresentationEngine's constraint solver/rule inter- 
preter; 

(14) Selective presentation can occur as a result of the 
previous step. A grid consistently preserves the overall 
presentation design; 

(15) aPresentationEngine realizes the concrete presenta- 
tion's (aConcretePresentation) by determining its final 
form including attributes and settings; 

(16) aPresentationEngine displays the concrete presenta- 
tion (aConcretePresentation); 

(17) aPresentationEngine notifies the application 
(anApplication) of its successful initialization; and 

(18) anApplication evokes aPresentationEngine's event 
handling routine. 

Software and Design^ 

Another exemplary embodiment of a service is in the 
context of the World Wide Web, and more specifically a 
corporate gateway web site will be used to further describe 
the architectural framework according to the present inven- 
tion. A corporate gateway web site may be designed to ser^e 
a company's online product and servicecatalogue, customer 
service center, or depending on the ^company's line of 
business, serve as a content navigator. In this exemplary 
embodiment, XYZ Communications-is ^communications 
company that has set up a,w eb sit ejhat-includes- corporate 
product and service information and serves as=a:gateway to 
aggregated- content (e.g., special events, community 
information; etc.). 

The present invention uses a basic structure called a 
feature-vector that consists of attribute-value pairs, e.g., 
"keywordocooking", or "author=Smith", etc. A feature in a 
feature-vector is represented by a type (e.g., keyword, 
geo-location, address) where the feature type encapsulates 
validation routines for authenticating the feature's data. 
These routines may be utilized by meta tools such as editors 
to validate the data entered at the interface. 
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A user model simply contains a feature -vector that is 
made up of a set of weighted features. Hie weight designates 
the relative importance of the feature with respect to a user's 
apparent interest. A feature and its associated weight may be 
explicitly or implicitly defined, i.e., manually set by the user, 5 
or derived by some statistical or machine learning algorithm 
analyzing a user's previous interaction history. Community 
models that represent a set of users may be created by 
bringing together users for different reasons (location, 
interest, job, or event). Therefore, a user model may actually 10 
represent an aggregate of several user models, each one 
representing a different persona, e.g., work, home, etc. as 
shown in FIG. 13. 

As previously discussed, a content model is required by a 
content assembly engine to put together a story tailored to a 15 
user's request and profile. This requires a content model to 
be able to associate the various content elements semanti- 
cally to form a story and to associate the content with user's 
preferences. In addition, a presentation generator (i.e., Pre- 
sentation Engine) needs to provide adaptive content presen- 20 
tation given the delivery context, including the end user 
devic^configuratioo, network bandwidth, etc. The content 
model-should be able to offer altemative:presentations of the 
content for the .presentation ^generator toselect from. 

In the architectural framework according to the present 25 
invention, a content element is defined as an object repre- 
senting a piece of information that can be presented via one 
or moremediatypesrFIG. 14 shows a diagram of semantics 
and content where the semantics describe what a content 
element is about. The semantics could^potentially.enable a 30 
content assembly engine to associate content elements on a 
more semantic level. An event or item on our exemplary web 
site could be represented as a content element that is media 
independent, but can manifest itself in multiple forms or 
representations such as a text document, an audio/video clip, 35 
or even a multimedia presentation. For example, assuming 
our exemplary web site included events such as information 
regarding a 1996 game, a baseball ad, and nature ad, FIG. 15 
displays how each of these events could have multiple 
representations of content. 40 

The application framework according_to the present 
invention uses dynamic content Assembly. With this 
approach, the development^ an-apphcation "or service is 
similar to the process 'of creating a dynamic story or movie 45 
that can adapt to a user, the available content, and the context 
at the time of delivery. The present invention iises7 among 
other concepts, three basic concepts in support of dynamic 
content assembly: siory^-filter, arid scene. 

A filter is a construct that takes in a se t of content elements 50 
and returns a subset of the original inputs. A filter-has? 
specific filteringsemantics, e.g., a feature-based filter that 
uses a feature (e.g., "keyword-television") to comb through 
an input set of content elements to retrieve content elements 
that match the feature. FIG. 16 shows an example of two 55 
such filters and the results being joined by an Andfilter that 
"ands" the results of two other filters. In this example, we 
have selected two content elements, one selected explicitly 
by a content ID and the other by filtering for advertisements 
that have been characterized to be related to nature. gQ 

By chaining filters, complex filtering patterns can be 
produced. A composite filter enables the creation of hierar- 
chical layered reusable content assembly. A scene is a 
composite filter that basically corresponds to one element 
and a story. By assembling a series of modular, layered 65 
scenes, we can tell a story at a fine level of granularity tuned 
to the user and the delivery context 
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The architectural framework according to the present 
invention uses adaptive presentation in that scenes are 
presented in different ways depending on the available 
context of delivery (such as available display real estate, the 
network connection, etc.). To support adaptive presentation, 
a presentation engine may generate presentations that take 
into account the context of delivery and select appropriate 
media representations to show the content element In the 
present invention a template, that acts as a proxy for a story 
element or scene element, is used regarding laying out and 
arranging the presentation elements. A primitive template 
has the responsibility of selecting the appropriate media 
element. A composite template serves to support the design 
of hierarchal presentations with a fine level of specification 
and control. By implementing these concepts and objects, 
the present invention supports the creation of custom pre- 
sentation components that are refinements of the basic 
presentation classes that can render a scene to a user in the 
most appropriate form. In the present invention, presentation 
components have the ability to render a scene without 
having to change the story. 

Application Subsystems 

Creating an application in accordance with the present 
invention involves interfacing to each subsystem's public^ 
interface. Each subsystem's public interface is encapsulated 
in the public operations of a select set of objects within each 
subsystem. An application is basically the glue that brings 
together invocations to the public interfaces of the sub- 
systems as well as to any other external subsystems, e.g., 
databases. FIG. 17 shows a diagram of an anatomy of an 
application using the architectural framework according to 
the present invention. Further, the following pseudo-code 
describes the basic framework of an application according to 
the present invention: 
Application: 

main( ) 

umMgr=new UserModelManager 
storyEngine=new StoryEngine(umMgr); 
storyEngine.init( ); 

presentationEngine=new PresentationEngine(umMgr); 

presentationEngine.intit( ) 

current_sceneoStartScene 

While Until exit( ) 

StoryEngine.assemble(current_scene) 
storyEvent»PresentationEngine.present(current_ 
scene) 

current_scene=StoryEngine.dispatchEvent 
(storyEvent) 
end while 
end main 

In summary, an application proceeds through the following 
steps: 

(1) Creating and initializing a UserModelManager; 

(2) Creating and initializing a story engine; 

(3) Creating and initializing a PresentationEngine; 

(4) Selecting a story element (i.e., scene) to be the initial 
element of the story; 

(5) Calling upon the StoryEngine to assemble a "story" 
given the initial element; 

(6) Upon the StoryEngine completing its assembly task, 
calling upon the PresentationEngine to present the 
"story"; 

(7) Dispatching a story-relevant event to the StoryEngine 
to determine the next story element (scene) to play; 
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(8) Based on the outcome of the event, set the next story application server and returns a generated presentation of the 

element (scene) to be assembled and subsequently application. For example, if the application resided on the 

presented. web server, a browser could serve as the user interface 

Referring to our exemplary web site example, the initial allowing the user to request a page for presentation 

story element is set to an element representing the home 5 (shipping along some form of identification, cookie, embed- 

page of XYZ Communications' web site. As the story plays ded CGI argument, etc.). The server would then assemble 

out with user interaction, the system proceeds through its and generate a complete presentation and return HTML that 

assemble -present-dispatch steps, a kind of dynamically gen- would be rendered in the browser, 

erated contextual movie. Therefore, a user could rapidly end i n terms of access, the interface may be a generic interface 

up, for example, on one page of the XYZ Communications 10 like an ITFML browser which only acts as an access point 

web site because a user has shown a continuing interest in and waits forTroinplete server-side generated presentation 

the subject matter of that page. This interest was detected to be rendered in its native HTML. Alternatively, in a web 

because the user has had a tendency to select information environment once again, a Java applet that only implements 

that can be described to have some sort of connection with a custom user interface may be downloaded. The applet 

that subject matter. For example, if the web page was a 15 would need to interface to the application server via some 

vegetarian page, the user may have shown interest in eating sor t of protocol so it could render a server-side generated 

healthier, therefore, a connection with healthy diets. The end presentation utilizing its native Java widgets, 

result is that the user would not have to wade through an FIG. 20 shows a network diagram of an exemplary 

extensive set of links and/or pages on topics of no or hide pecr . to .pe e r distributed system. Ideally, all components may 

interest to him or her . 20 be distributed across the network in principle. For a variety 

Regarding the interface m this example the user is rf reasons> L load balancing , low bandwidth, intermittent 

interacting with a user interface or a ^browser depending on network connections> cffidcnt rcS ource utilization, etc., situ- 

the implementation environment. Additionally the Story ^ ^ ^ ^ warranl configuring an applica- 

Engine and the Presentation Engine serve as single points of tfon in a M ^buted architecture (e.g., CORBA, Java 

mterface tc . the story and presentation databases respec- 25 RMI). This partitioning implies that the application may be 

tively. The User Model Manager takes on a similar role over reduced {Q mterfad to clieQts mat do me real work 

tiie database of user models by being a gateway to any user of ta]king to theif rcspective com p 0 nents. In a truly distrib- 

orma ion. uted svstem a component may potentially take on both roles 

System Partitionin °^ sctver client. Regarding access, in this configuration, 

^ * 30 the point of access is dependent on the implementation 

Typically, an application is not just resident on one and/or delivery environment, 
processing element but is distributed or networked, i.e., 

distributed and partitioned in multiple elements. The fol- Object-oriented Base Framework Design 

lowing shows embodiments of an application developed 3S ^ following m descriptions of exemplary class dia- 

with the architectural framework according to the present grams md basc dasscs ^ bc uscd in m applicatio n S 

invention where the application is partitioned across net- fr^ynvk according to the present invention, 
work elements. 

FIG. 18 shows a network diagram of an exemplary thick User Modeling Subsystem Classes 

client-thin server design embodiment according to the ^ VTn u , . r « 

present invention, lie client is bundled with both runtime 40 FI °. 21 f°T an «™»pl«y~ ™™W class diagram 

engines (Le., Story Engine, Presentation Engine) and the to * e P 165 ? 1 ™ c ?' f ^ l °^ £ 0V £ 6 

User Model Manager that interfaces to a database of user de SCT1 PU°ns°i^ exemplary basc classes shown in FIG. 21. 

« j i tu * * . j The user modeling subsystem may be a collection of classes 

models. The story, content, and presentation databases are iL 4 _^ iL j • * 11 

™ . 1 u a Tn.- • .u r • j .u that supports the creation and maintenance of user models 

remotely based. This requires the Story Engine and the 45 a\ \ 

Presentation Engine to be designed to hide the details of UserModel 

accessing remote databases, similar to the role of the User n . . 

Model Manager, which serves as a gateway to a repository r^? 0 "? ° n * . . 

r J Ti 1 1 * w *t_ / This class represents a user s mterests through a Fea- 

of user models, local or remote. Moreover, the remote t 17 : . , • j j^ .j. 

, ( M . ' . . . „ ' „ # ture Vector. Features are a content-independent metadata 

databases need to be managed by server processes that can cn . . , y . ± . A 

» .* ~ n- 1 ~ * ~ , ./ • , r t 50 structure that serves as a common denominator between 

serve multiple remote users and provide an mterface to , 4 A 

r * * u- * • /• 1 * users and content, 

chents for remote object communication (i.e., sockets, ~ ...... 

Java's RMI (Remote Method Invocation), CORBA KesponsiDilities: 

(Common Object Request Broker Architecture), etc.). PersisteDt re P' esen ^ on ° f «™ interests - 

From the access perspective, this particular design 55 Private Properties, 

requires the client to be either: resident on the client's user_id: string«nuU 

machine, or downloaded at the point of remote access, e.g., A strin S Damc for a UserModel, 

Java applet. features: Feature Vector=null 

FIG. 19 shows a network diagram of an exemplary thin A set of features representing a user's weighted inter- 
client-thick server design embodiment according to the 60 A common denominator between UserModels 
present invention. With a thin client, the majority of the . md ContentElements. 
application resides on the server side. Whether the complete Public Methods: 
application resides on the server or not depends on the UserModel (uid: String-null): 

implementation of the user interface and the choice of Public constructor parameterized for a string-based 

delivery environment. Regardless, the interface needs to 65 user ID, 

have the capability to access and operate the application by deleteFeature (feat: Featureonull): void 

sending a request to the remote host, who in effect acts as an Delete a Feature from UserModel's Feature Vector. 
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addFeaturc (feat: Feature=null): void 

Add a Feature to the UserModel's Feature Vector 
findByType (sname: String-null): Feature Vector 
Find all Features present in the UserModel's Fea- 
tureVector of the indicated type (i.e., typename) and 5 
return the results in a new Feature Vector. 
findEntry (feat: Feature=null): Feature 
Find supplied Feature in the UserModeFs Feature Vec- 
tor 

similarity (features: Feature Vector-null): float 

Compute a numerical score indicating the degree of 
similarity between a UserModel (its FeatureVector) 
and the supplied FeatureVector. 
read (istream: InputStream=null): void ^ 
Read a UserModel from a InputStream (language- 
specific). 

write (ostream: OutputStream=null): void 
Write a UserModel to a OutputStream (language- 
specific). 20 
Community 
Description: 

This class represents a set of users as a community. The 
inherited FeatureVector from the UserModel base class is 
treated as stereotype user of the community and computed 25 
by the Community class. 

Communities can be created explicitly or implicitly. 

Responsibilities: 

Maintain a set of users. 

Maintain a stereotype of the user. 30 
Derived from UserModel 
Private Properties: 
users: UserModel-null 
A set of users, i.e., UserModels. 
Public Methods: 

Community (id: String-null): 

Public constructor parameterized for a string name. 
addUser (user: UserModel ^default): void 

Add UserModel to set of UserModels. 
deleteUser (user: UserModel-null): void 

Delete UserModel from set of UserModels. 
getUM (uid: String=null, umMgn UMMgr=null): void 



35 
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Retrieve a UserModel through the UMmgr and cache 



45 



the UserModel in the Community. 
getAll (umMgr: UMMgr-null): 
Retrieve all contained UserModels through a UMMgr 
and cache them in users. 
UMmgr 50 
Description: 

This class serves as interface to all UserModels and 
Communities. It hides all remote access to remote models. 
Responsibilities: 

Maintain a global set of user models. 55 
Access point to a all user models. 
Private Properties: 
hostID: String-null 

String ID for remote or local host system. 
baselD: String=null 

String ID for root community (global community that 
contains all models), 
base Community: Community-null 

The root Community for all models. 65 
Public Methods: 

UMmgr (hostid: String=null, baseid: String=null): 



Public constructor parameterized for string ID for the 
remote host home to the UserModels, and string ID 
for the root Community. 
init( ): void 

Initializes the UMmgr's internal data. 
getUM (uid: String-null): UserModel 

Retrieve a UserModel, transparently from a remote or 
local host system. 
saveUM (um: UserModel=null): void 

Save the UserModel transparently to a remote or local 
host. 

deleteUM (um: UserModel=null): void 

Delete UserModel from the pool of UserModels at a 
remote or local host. 
generateStereotype (cm: Community-null): UserModel 
Based on a set of UserModels, generate a UserModel 
that typifies a user in the given Community, 
getcommunity (uid: String-null): Community 
Retrieve a Community model, transparently from a 
remote or local host system. 
saveCommunity (cm: Community-null): void 

Save the Community model transparently to a remote 
or local host. 
deleteCommunity (cm: Community=null): void 

Delete Community model from the pool of UserModels 
at a remote or local host. 
UserHistory 
Description: 

This class represents a repository of a chronically-ordered 
set of StoryEvents as a result of user interaction. 
Responsibilities: 
Maintain a set of StoryEvents. 
Private Properties: 
events: Set of StoryEvent-mill 

the set of StoryEvents that have occurred as a result of 
a specific user's interaction. 
Public Methods: 
UserHistory (uid: UserModel null): 
Public constructor parameterized for a string name 
indicating the ID of the UserModel. 
addEntry (event: StoryEvent=null): void 

Add an entry, i.e., StoryEveot, to the history. 
deleteEntry (event: StoryEvent=null): void 

Delete StoryEvent from the history of events, 
purge (purgeDate: Date-null): void 

Remove all StoryEvents from the history that occurred 
before the indicated date, 
read (istream: InputStream=null): void 
Read UserHistory from a InputStream (language- 
specific). 

write (ostream: InputStreamonuU): void 

Write a UserHistory to a OutputStream (language- 
specific). 
AnalysisWorkbench 
Description: 

This class brings together an array of analysis tools to 
update and better target UserModels and extract communi- 
ties of interest. 

This component is basically a learning system. 

Responsibilities: 

Update UserModels based on their UserHistories 
Compute stereotypical users for Community models. 
Compute correlations between features in UserModels 
Compute clusters of users to discover implicit communi- 
ties of interest. 
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Public Methods: 

AnalysisWorkbench( ): 
Public constructor. 

computeStereotype (cm: Community =null): Feature Vec- 
tor 

Compute stereotypical user, i.e., a Feature Vector, for a 
given Community, 
reduce (history: UserHistory-null): Feature Vector 

Reduce a UserHistory to a Feature Vector. 
A UserHistory contains StoryEvents where some in turn 
carry information as a result of the user selecting a Conten- 
tElement. 

This information serves as raw data to determine the 
effectiveness/relevance of a UserModers features 

cluster (users: Community«null): Community 
Apply cluster analysis to a Community of users and 
generate a Community of Communities, each repre- 
senting a cluster. 
Story Engine Subsystem Classes 

FIG. 22 shows an exemplary StoryEngine class diagram 
according to the present invention. The following provide 
descriptions of exemplary base classes shown in FIG. 22. 
The Story Engine may consist of a content assembler (the 
Story Engine itself) and the databases containing data struc- 
tures that specify an application and the underlying content 
model that represents and interfaces.to-mult^leTepresenta- 
tions of multimediaj^ntent^elein^nts^-r^^^ 
Story Element" — — 

Description: 

StoryElement is the abstract class for all components that 
makeup a story, i.e., elements to be assembled dynamically. 
Responsibilities: 

Abstract base class with an associated string ID. 
Private Properties: 
name: String«null 

The name of the StoryElement. 
Public Methods: 

StoryElement (sname: String=null): 

Public Constructor for Story Element. 
getName( ): String 

Return the name of StoryElement. 
read (istream: InputStream-null): void 

Read a StoryElement from a InputStream (language- 
specific). 

write (ostream: OutputStreamonull): void 

Write a StoryElement to OutputStream (language - 
specific). 

Filter- 
description: 

A loiter is a StoryElement that basically takes a set of 
input ConfentElements and outputs a subset of the Conten- 
tElements based^on the Filter's filtering semantics. 

Each ContentElement that ends up in the Filter's set of 
outputs potentially has a set of points of interaction called 
anchors. These anchors can be activated as a result of, e.g., 
of user interaction, and produce an application event called 
a user selection. So, a Filter's anchors are derived from its 
contained ContentElements' anchors. 

Responsibilities: 

Filter a input set of ContentElements based some filtering 

semantics (as defined by concrete subclasses). 
Handle StoryEvent dispatched from the StoryEngine. 
Derived from StoryElement 
Private Properties: 
inputs: ContentElement=null 
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The set of ContentElements passed into the Filter for 
evaluation. 

The set can be a selective set as bound by the application or 
the global set, which is all existing ContentElements in the 
current database, 
outputs: ContentElement«null 
The set of ContentElements resulting from evaluating 
the Filter. 
maxOutputs: Integer=-1 

Indicates the maximum number of ContentElements 
stored in the outputs upon evaluating the Filter, 
anchors: Set of Anchors=null 

The set of anchors associated with the ContentElements 
enumerated in the outputs. 
Public Methods: 
Filter (sname: String=null): 

Public constructor for Filter parameterized for the 
string name, 
addlnput (iElement: ContentElement-null): 

Add a ContentElement to the set of inputs, 
addinputs (iElements: Set of ContentElement=de fault): 
void 

Add a set of inputs to the set of inputs, 
getourput (i: int=0): ContentElement 

Retrieve the ith ContentElement from the set of out- 
puts. 

getOutputs( ): Set of ContentElements 

Retrieve the Filter's complete set of Outputs. 
getElementNames( ): Set of String[ ] 

Get an array of ContentElement names. 
setMaxOutputs (max: Integer=-1): void 

Set the maximum of ContentElements cached in the 
outputs. 
apply( ): boolean 
Apply the Filter's filtering semantics and place results 
in the outputs. 
handleEvent (event: StoryEvent null): boolean 
This operations handles any Application-level event that 
has been dispatched by the StoryEngine. 
Private Methods: 

addoutput (ce: ContentElement-null): void 

Internal operation for adding a ContentElement to the 
set of the outputs. 
removeOutput (element: ContentElement null): void 
Internal operation for removing a ContentElement from 
the set of outputs. 
CollectionFilter 
Description: 

This type of Filter serves as the base class for all 
collection-oriented filters. Specialized classes of Collection- 
Filter define specific filtering semantics. 

Responsibilities: 

Operate over a set of contained Filters. 
Derived from Filter 
Private Properties: 
subFilters 

The collection of contained Filters. 
Public Methods: 

CollectionFilter (sname: String-null) 

Public constructor parameterized for string name. 
addFilter (filter Filter-null): void 

Add a Filter to the collection of Filters, i.e., subFilters. 
removeFilter (filter: Filter=null): void 

Remove a Filter from collection of Filters, i.e., subFil- 
ters. 
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getFilter (i: integer«-l): Filter Scene 

Get the ith filter in the subFilter set. Description: 

FeatureFilter This is a composite Filter composed of other Filters, This 

Description: Filter provides the capability to construct hierarchically 

This class is a Filter whose filtering semantics is to filter 5 layered set of Filters and their associated ContentElements. 

the input set of ContentElements based on the supplied Evaluating a Scene results in its outputs residing in the 

Feature. The resulting set of matches is stored in the outputs. outputs of the contained filters. This is one of the main 

The maximum number of matches is set in maxOutputs. differences between a Scene and other CollectionFilters. 

Responsibilities: The second main difference is that a Scene is the only 

Filter a set of ContentElements using a Feature. 10 presentable Filter. In order for any Filter to be presented, it 

Derived from Filter mu st be embedded in a Scene. 

Private Properties: This Filter supports the abstraction of services, presenta- 

feature: Feature-null tions * 

Filtering pattern. Responsibilities: 

Public Methods: 15 Aggregates a set of filters. 

FeatureFilter (sname: String-null, feat: Feature-null): Interface for presenting a story. 

Public constructor parameterized for string name and a Derived from CollectionFilter 

Feature. Private Properties: 

setFeature (f: Feature-null): void jo presentation: Presentation-null 

Set the Filter's feature to use as a filter pattern. The presentation responsible for rendering the Scene. 

UserFilter Public Methods: 

Description: Scene (sname: String-null): 

This Filter interfaces to the current user's user model (as Pub he constructor parameterized for string name, 

specified by the StoryEngine). It utilizes the user model's ^ se tPresentation (p: Presentation-null): void 

feature vector to filter content elements from the input S et the dependent Presentation of this Scene. 

ContentElements. AndFilter 

Responsibilities: Description: 

Filter a input set of ContentElements using a UserModeL The AndFilter is basically a union set operator. It takes 2 

Derived from Filter 30 or more filters. The combined results of the evaluation of this 

Private Properties: Filter are stored in its outputs, 

user: UserModel=null Responsibilities: 

References current user model. ANDing the results of 2 or more contained Filters. 

Public Methods: Derived from CollectionFilter 

UserFilter (sname: String-null, um: UserModel«null): 35 Pub^c Methods: 

Public constructor parameterized for string name and a AndFilter (sname: String-null): 

UserModeL Public constructor parameterized for a string name. 

setUserModel (um: UserModel=null): OrFilter 

Set the UserModel for the Filter. Description: 

RuleFilter The OrFilter is basically a union set operator. It takes 2 or 

Description: more filters. The combined results of the evaluation of this 

This class is a rule-based filter that applies a predicate Filter are stored in its outputs, 

operation, that in turn applies either a THEN or ELSE filter. Responsibilities: 

Responsibilities: 45 ORing the results of the 2 or more contained Filters. 

Apply a predicate operation and branch to one of two Derived from CollectionFilter 

filters. Public Methods: 

Derived from Filter OrFilter (sname: String-null): 

Private Properties: Public constructor parameterized for a string name. 

thenFilter: Filter=null so TemporalScene 

If predicate results in true, apply thenFilter. Description: 

elseFilter Filter«null Scene nas a special capability to sequence its con- 

If predicate results in false, apply the elseFilter. ^ Filte f asso <? ated ContentElements i.e outputs. By 

Public Methods' defining a playout duration, each contained Filter s Conten- 

_ * . « tElements will be presented one at a time. Optionally, the 

RuleFilter (sname: Stnng-null, tFilter: Filter-null, eFil- temporal playout can be repeated for a specified number of 

ter: Filter-null): ^ ■ * 

^ trUCt0 7 ar ^ m cl er ^ ed f0f 4 Strfng mme ' a Ie <l™ es a Presentation be able launch a timer 

I HEN niter, and a ELSE filter. ^ ultimatel y returns a TimeoutEvent to this Scene via the 

setTHEN (filter: Filter=null) StoryEngine. 

Set the THEN filter. Responsibilities: 

setELSE (f : Filter-default) Specifies a temporal playout of the resulting set of filtered 

Set the ELSE filter of the RuleFilter. ContentElements. 

predicate( ): boolean Derived from Scene 

This operation executes its code and returns a boolean 65 Private Properties: 

result. This operation needs to be redefined by con- duration: TimeUnh-null 

crete subclasses. The duration of the Scene. 
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repeating: boolean 

Indicate if Scene will repeat its temporal playout. 
numReps: integer— 1 

Indicates the number of repetitions of the playout 
Public Methods: 

TemporalScene (sname: String=null, interval: TImeUnit= 
null, repeat: boolean=false, numreps: integer=-l): 
Public constructor parameterized for time interval, 
indication if allowing repetitions, and the number of 
repetitions. 
setDuration (time: TimeUnit-null): void 

Set the duration of playout for each contained Conten- 
tElement. 
repeating (flag: boolean=null): void 

Indicates if the temporal playout will be repeating. 
setReps (reps: integer«-l): void 

Set the number of repetitions of the temporal playout. 
StoryEvent 
Description: 

This class represents events of interests to StoryElements. 
The PresentationEngine is responsible for listening for 
events. Any StoryEvents are forwarded to the StoryEngine 
and ultimately to the Scene and the appropriate sub- 
components. 

Responsibilities: 

Represent the event of user or story action (timer) that 

occurred on a ContentElement. 
Private Properties: 
timestamp: Timestamp=null 

Indicates the time of event occurrence. 
Public Methods: 

StoryEvent (tstamp: TimeStamp=null): 
Public constructor parameterized for timestamp. 
UserSelectionEvent 
Description: 

This event occurs when a user activates an Anchor. 
Responsibilities: 

Represent a user action of selection. 
Derived from StoryEvent 
Private Properties: 
anchor Anchor=null 

Indicates selected Anchor. 
Public Methods: 

UserSelectionEvent (a: Anchor=null): 

Public constructor parameterized for a Anchor, 

getAnchor( ): Anchor 
Retrieve Anchor object associated with this event. 
TimeoutEvent 

Description: 

This event occurs when a timer has expired. A timer is 
called for by a TemporalFilter and is realized by a 
Presentation-side Timer object. 

Responsibilities: 

Represent a timeout event for a ContentElement. 
Derived from StoryEvent 
Private Properties: 
scene: TemporalScene -null 
Indicates the original TemporalScene that initiated the 
timer request that has now expired. 
Public Methods: 
TimeoutEvent (f: Filter=null) 

Public constructor parameterized for a filter (i.e., 
TemporalScene) . 
StoryEngine 
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Description: 

This class is the system-level interface to the story sub- 
system. 

Responsibilities: 
5 1. Execution engine for assembling StoryElements based 
on a Story Model. 

2. Maintains history of played Filters. 

3. Tracks current Filter. 

10 4. Interfaces to the UserModeling system. 

5. The StoryEngine also interfaces to the global pool of 
ContentElements. 
It supplies these elements by default to the Filters' inputs. 
Private Properties: 
15 currentFilter: Filteronull 

Indicates currently executing Filter. 
umMgr: UMMgr-null 

Access to a UMMgr that interface to the TJserModel 
pool. 

20 playHistory: Set of Filter-null 

The set of Filters played out during the session. 
Public Methods: 

StoryEngine (umMgr: UMMgr=null): 

Public Constructor parameterized for a UMMgr. 
assemble (scene: Scene-null):, boolean 
Startup the composition of the current Scene. It in turn 
calls the Scene's evaluate operation that triggers the 
Scene recursively to trigger the evaluation of its 
30 contained Filters. 

dispatchEvent (event: StoryEventonull): Scene 

Dispatch the StoryEvent, originally forwarded by the 
PresentationEngine, to the current Scene. This 
operation calls Scene's handleEvent operation. 
35 This operation ultimately returns a new Scene that the 
StoryEngine executes to continue the playout of the 
story. 
init( ): boolean 

Initialize StoryEngine including: 
40 Load the "story database" 

Retrieve a UserModel from UserModelManger. 

Anchor 
Description: 

This class is representing an anchor that links to a source 
45 Filters and sink Filter. The location of anchor is set in the 
sourceFilter attribute. Its destination is set in the destina- 
tionFilter attribute. 

Most importantly, the destination can be determined at 
playout time, i.e., run-time. 
50 Responsibilities: 

Maintain a link between two Filters 
Private Properties: 
sourceFilter: Filter=null 

This is the source Filter of the Anchor. 
55 destinationFilter: Filter-null 

This is the sink Filter for the Anchor. 
Public Methods: 

Anchor (anchorName: String-null): 
6Q Public constructor parameterized for string name. 

Anchor (anchorName: String=null, srcFilter: Filteronull, 
dstFilter: Filter-null): 

Public constructor parameterized for a string name, a 
source Filter, and a destination Filter. 
65 setSource (source: Filter-null): void 
Set the source Filter of the Anchor. 
setDestination (dest: Filter=null): void 
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Set the destination Filter of the Anchor. 
getSource( ): Filter 

Get the source Filter. 
getDestination( ): Filter 

Get destination Filter, 
read (istream: InputStream=null): void 

Read an Anchor from a InputStream (language - 
specific). 

write (ostream: OutputStream=null): void 
Write a Anchor to a OutputStream (language-specific). 
Presentation Engine Subsystem Classes 

FIG. 23 shows an exemplary PresentationEngine class 
diagram according to the present invention. The following 
provide descriptions of exemplary base classes shown in 
FIG, 23. The Presentation Engine may consist of a presen- 
tation generator, and a library of presentation components 
that may be matched up with the corresponding application 
elements (i.e., story elements) that will compute the final 
presentation form of the content elements. 
Template 

Description: 

This is an abstract class that maps to a StoryElement. This 
class needs to be specialized to define appropriate presen- 
tation properties in accordance with the target delivery 
platform (e.g., HTML Template). 

Responsibilities: 

The primary responsibility of Template is to determine 
which representation (ContentMediaElement) of the 
associated ContentElement to render. 

Private Properties: 

contentElement: ContentElement=null 

The ContenetElement to be presented/rendered. 

currentRepresentation: ContentMediaElement=null 
Currently selected representation of the associated 
ContentElement. 

name: String=null 

string-based identification of the Template. 

Public Methods: 

Template (contentElement: ContentElement-null, con- 
text: PresentationContext-null): 
Public constructor parameterized for its associated Sto- 
ryElement and a PresentationContext. 
initialize( ): void 

Initialization sets the Template ready for generation. 
A Template can be called upon successively to regen- 
erate itself and select an alternative ConteutRepre- 
sentation. 
render( ): void 

Format or display the final form of the Template to the 
target environment, 
select (context: PresentationContext=null): ContentRep- 
resentation 

Heuristic-based selection of a ContentRepresentation 
from the Template's associated ContentElement's 
pool of representations Selection based on original 
design intent and the PresentationContext. 
generate( ): boolean 
Top-level operation to generate a candidate form of the 
Template. This operation calls upon select( ). 
Returns true if successful. 
evaluate( ): boolean 

Given the current PresentationContext, this operation 
evaluates the Template in its candidate form to 
determine if its acceptable, 
read (istream: InputStream=nuU): void 
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Read a Template from InputStream (language -specific), 
write (ostream: Outputs tream-null): void 
Write a Template to OutputStream (language-specific). 
CompositeTemplate 
5 Description: 

This class is an aggregate that maintains and represents a 
set of Templates. This class enables hierarchical-structured, 
recursive, presentations. This class typically maps to a Scene 
in the StoryElement domain. CompositeTemplate redefines 
10 render, select, generate, and evaluate operations. 
Responsibilities: 

Represent and manage the final form of the contained 
Templates. Calls upon contained Templates to itera- 
tively generate their final form to satisfy the design 
15 intent and constraints of the CompositeTemplate. 

Working with a LayoutElement computes candidate lay- 
out of the contained parts. 
Key Capability: 
2Q Embodied with "smarts" to join or split contained original 
set of Templates depending on the design intent of a 
subclass. This capability is a cooperative process with 
a LayoutElement who is a spatial layout expert. 
Derived from Template 
25 Private Properties: 

subTemplates: Template-null 

The set of contained Templates that a Composite Tem- 
plates manages, 
scene: Scene =ou 11 
30 Associated Scene object. 
Public Methods: 

CompositeTemplate (scene: Scene =null, name: String- 
null, context: 
35 PresentationContext=null): 

Public constructor parameterized for a Scene, string 
name, and a PresentationContext. 
layout (layoutElement: LayoutElement=null): boolean 
Computes the layout of its contained templates in 
40 cooperation with a LayoutElement. The Composite- 

Template delegates to a LayoutElement the abstract 
task of computing a constraint-based layout, i.e., 
determines how to glue the content elements 
together, while the CompositeTemplate has the spe- 
45 cific task of dictating a specified design style. 

addTemplate (tmpl: Template=null): void 

Add a Template to set of subTemplates. 
deleteTemplate (tmpl: Template=null) 

Delete a Template from the set of SubTemplates. 
50 Presentation 
Description: 

This class encapsulates the rendering of a presentation of 
a Scene. 
Responsibilities: 
55 The primary responsibility of the Presentation class is to - 
create the corresponding presentation object hierarchy 
that map the hierarchical structure of a Scene object 
Private Properties: 
60 rootScene: Scene=null 

The top-level scene associated with a Presentation. 
rootTemplate: CompositeTemplate-null 
The top-level CompositeTemplate associated with the 
rootScene. 
6S Public Methods: 

Presentation (scene: Scene = null, context: 
PresentationContextonull, id String=«null): 
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Public constructor parameterized for a Scene, a name, 
and a Presentation Context, 
map (scene: Scene-null): CompositeTemplate 
This operation basically constructs a tree comprised of 
templates that map to each StoryElement contained 5 
in a Scene and its subcomponents. 
This operation returns the root CompositeTemplate that 
maps to the root Scene. 
render( ): void 
This operation in turn calls renders on its contained 10 
Templates. 
generate( ): boolean 
This operation in turn calls generates on all contained 
templates to launch the generation of a Presentation. 15 

Timer 
Description: 

Ihis class presents a timer entity for showing a Presen- 
tation for a specific interval of time. A timeout event is 
spawned when a Timer has expires. 

Responsibilities: 

Represent a timer that spawns a timeout event. 
Private Properties: 
duration: TimeUnit»null 

Duration of timer 25 
Public Methods: 

Timer (presentation: Presentationonull, interval: 
HmeUnitonull): 

Public constructor parameterized for a Presentation and 
a length of duration. 30 
setDuration (interval: TimeUnit=null): void 
Set the duration of the Timer. 
LayoutElement 
Description: 

This is an abstract base class that is intended to coordinate 35 
the arrangement of the elements that makeup a Composite - 
Template. 

Concrete subclasses need to define the appropriate opera- 
tions and attributes dependent on the specific delivery envi- 
ronment (e.g., HTML, X Windows, set-top, etc.). 40 

Responsibilities: 

Spatially arrange a CompositeTemplate 's elements. 
Private Properties: 

cTemplate: CompositeTemplate=null 45 
the CompositeTemplate whose elements are being 
arranged. 

pContext: PresentationContext-null 
current PresentationContext. 

Public Methods: 50 

LayoutElement (CompositeTemplate: 
CompositeTemplate=null): 

Public constructor parameterized for a CompositeTem- 
plate and a PresentationContext. 
arrange (cTmpl: CompositeTemplate-null): boolean 55 
This operation computes the spatial arrangement of a 
CompositeTemplate's elements. 
PresentationEngine 
Description: 

The PresentationEngine is the system-level interface to 60 
the presentation system. 
Responsibilities: 
Determine PresentationContext. 

Find most appropriate matching Presentation for the 65 

incoming Scene. 
Hand off a StoryEvents back to the Application. 
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Private Properties: 

PresentationContext: PresentationContext=null 
The current PresentationContext for the given Presen- 
tation. 

presentation: Presentation^null 

The current Presentation being generated/presented. 
Public Methods: 

PresentationEngine (userModelMgr: UMMgr»null): 
Public constructor parameterized for a UserModelMa- 
nager. 
init( ): void 

This operation loads the presentation database. 
present( ): StoryEvent 

This operation includes the following steps: 

1. Invoke Scene lookup operation. 

2. Generate the currentPresentation. 

3. Wait for a StoryEvent and return it. lookup (scene: 
Scene=null): Presentation 

This operation attempts to match the scene from the 
StoryEngine to a corresponding Presentation that 
will have the most appropriate mapping to the Scene 
components. 
handleEvent( ): StoryEvent 

This operation waits for a StoryEvent to be detected 
and subsequently returned to the application. 
PresentationContext 
Description: 

This class represents the current snapshot of the delivery 
environment at any given moment during the generation of 
a Presentation. This component is like the UserModel is to 
the user's profile, as a PresentationContext is to a profile of 
the presentation environment. 

Responsibilities: 

Maintain a collection of attribute-value pairs that describe 

the delivery environment. 
Private Properties: 

featureVector: Feature Vector 
Public Methods: 

PresentationContext (id: String«null): 

Public constructor parameterized for a string name. 
Content Model Classes 

FIG. 24 shows an exemplary content class diagram 
according to the present invention. The following provide 
descriptions of exemplary base classes shown in FIG. 24. 
ContentElement 

Description: 

ContentElement is the root class of the content model 
hierarchy. This class abstracts an element of content and 
maintains a set of ContentMediaElements, where each Con- 
tentMediaElement is a different representation of the Con- 
tentElement (e.g. text, image, etc.) 

Responsibilities: 

Multi-model representation of a element of content. 
Integrate multiple representations of a element of content. 
Private Properties: 
name: String-null 

the name of the ContentElement. 
keywords: Feature Vector=null 

utilizing feature vector to represent the semantic mean- 
ing. 

representations: Set of ContentMediaElement=null 

different media representations of the ContentElement. 
anchors: Set of Anchor =null 
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Public Methods: 

ContentElement (cname: String-null): 
Constructor 1 

ContentElement (cname: String «null, keys: String[ 

]=null): 

Constructor 2 
setKeywords (keys: String-null): void 

Set the keywords of this ContentElement 
getKeywords( ): Feature Vector 

Get the feature vector of the ContentElement. 
addMediaElement (cme: ContentMediaElement=null): 

void 

Add a ContentMediaElement the ContentElement's set 
of media elements. 
removeMediaElement (cme: ContentMediaElement- 
null): void 

Remove the given ContentMediaElement from the 
ContentElement's set of media elements. 
getName( ): String 

Return the ContentElement's name, 
read (istream: InputStreanionull): void 
Read a ContentElement from a InputStream (language- 
specific). 

write (ostream: OutputStream-null): void 
Write a ContentElement to a OutputStream (language- 
specific). 

getMedia (i: integer=-l): ContentMediaElement 

Get the ith ContentMediaElement contained in the set 
of representatioDS. 
Composite Content 
Description: 

This class supports the creation of an aggregation of 
ContentHements. 
Responsibilities: 

Maintaining a set of ContentElement. 
Derived from ContentElement 
Private Properties: 

components: Set of ContentElement-null 
Public Methods: 

Composite Content (cname: Stringonull, keys: Stringf 
]=null, cmpnts: Set of ContentElement»null): 
Public constructor parameterized for a string name, a 
set of keywords, and a set of subcomponents. 
addComponent (c: ContentElement-null): void 
Add a ContentElement to this CompositeContent's set 
of components. 
getComponent (i: int=0): Content 

Get the ith ContentElement contained in the compo- 
nent. 

removeComponent (cmpnt: ContentElement-null): void 
Remove a component from the components list. 

getComponentNames( ): String[ ] 

Get an array of component content names 
ContentMediaElement 

Description: 

This is a_virtual-clasrthat defines general attributes and 
operations^for ContentMediaElements. It will be imple- 
mented in Audio, Video, Image and Text subclasses. 

Tnis class basically acts as a wrapper class to media 
assets, hiding the details of the raw media. 

Responsibilities: 

Representation of a media asset (e.g., image, video 

segment, text segment). 
Private Properties: 
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name: String«=null 

The name of the presentation 
author: String-null 

The author of the presentation 
anchors: Set of Anchor=null 

The set of associated anchors 
Public Methods: 

ContentMediaElement (cmeName: String, cmeAuthor: 
String): 

Public constructor parameterized for a string name and 
string author's name. 
show( ): void 
Show the ContentMediaElement. 
Metadata Model Classes 

FIG. 25 shows an exemplary metadata class diagram 
according to the present invention. The following provide 
descriptions of exemplary base classes shown in FIG. 25. 
FeatureType 
Description: 

This is the abstract base class for all FeatureTypes. 
FeatureType is a wrapper class that encapsulates one or more 
primitive datatypes that collectively provide more meaning. 

For example, a FeatureType called homeLocation com- 
prised of 4 strings that represent street address, city, state, 
and country of a user, or geoLocation comprised of two real 
numbers that represent the latitude and longitude of a 
location. 

Responsibilities: 

Abstract base class for types of features. 
Private Properties: 
typeName: Stringonull 

The string name of this FeatureType. 
Public Methods: 

FeatureType (sname; String null): 

Public constructor parameterized for a string name, 
equals (object: Object=null): boolean 

Determines if object is of a specific FeatureType. 
validate( ): boolean 

Determines if the FeatureType associated data is valid, 
e.g., string within length bounds, 
read (istream: InputStream -null): void 

Read a FeatureType from a InputStream (language - 
specific). 

write (ostream: Outputs tream°nuli): void 
Write a FeatureType to OutputStream (language- 
specific). 

Feature 
Description: 

This class represents a weighted data, more specifically, a 
weighted FeatureType instance. The assumption is that the 
"data" is rated in terms of importance on a real number scale 
from 0.0 to 1.0. 

Responsibilities: 

Represent a weighted attribute of interest. 
Private Properties: 
weight: =»-l 

The weight of importance/priority on a real number 
scale from 0.0. to 1.0. 
data: FeatureType-null 

The FeatureType instance that comprises the Feature. 
Public Methods: 

Feature (data: FeatureType«null, wt: float-null): 

Public constructor parameterized for a datum and a 
weight. 

setWeight (wt: float»-l): void 
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Set the weight of the Feature. game history recap. The development team develops a story 

get\Veight( ): float model that further expands the block diagram shown in FIG. 

Get the feature's weight. 27. Each game review is developed as a separate component 

getData( )• Object therefore, each element is a self-contained independent 

Get encapsulated FeatureType's data 5 a gg re S ate of information. The architectural framework 

read (istream: InputStream-null): void a £°"*e to ^ P resent ™ cnt ™ sl ?P or4 f thc ^To^ 

„ j r . c , i0 / „ . of bottom up services and reusable story elements. The 1995 

Read a Feature from a InputStream (language-specific). . r .« . . r J e 4 , 

. «v .f game review will consist of a summary of the 1 995 game 

write (ostream: OutputStream-null): void and an advertisement that best matches the current user's 

Write a Feature to a OutputStream (language-specific). w uscr modeL ^ m6 gamc rcyiew wiu of a sum . 

FeatureVector mary of mc 1996 game and ^ advertisement that best 

Description: . . . ^ . , . matches the current user's user model but that has not been 

This class is a set of weighted Features. Tins class is a previously rev iewed by the user in the current session, 

basic data structure for representmg metadata. otherwise, an advertisement that simply is sports related. 

Responsibilities: 15 piG. 28 shows the resulting story model. By using 

Maintains a set of features. FeatureFilters, one can select specific content elements by 

Private Properties: referring to their contented. U^rHlters-filtcr:content ele- 

features: Set of Feature-null ments that match a user's usermodel,„retuming a prioritized 

A set of features. set of content elements sorted'by one level of similarity. 

Public Methods: 20 Since a story model is developed, the next step is to develop 

Feature Vector( ): or reuse presentation components that will be designated to 

Public constructor present the simple story. 

addEntry (feat: Feature-null): void m order to illustrate the presentation aspects of the web 

Add a Feature to the Feature Vector's set of Features. service example, a set of presentation templates that encap- 

deleteEntry (feat: Feature=null): void 25 sulate the HTMLdelivery environment are developed. In the 

Delete Feature from FeatureVector's set of Features. presentation domain, there are two types of presentation 

findEntry (feat: Feature-null): Feature a * mplate and f a CompositeTemplate. A Com- 

FmdmeFeaturemmeFeatureVector'ssetofFeatures. P 0SlteT ^Pi ate represents a set of Templates. Non- 

...... „ ... _ composite Templates are mapped to Content Elements. 

similarity (feat: Feature-null): float ^ Composite Templates are mapped to Scenes (a composite 
Compute numerical score indicating how similar/ ContentElement). In the creation of a presentation, Tem- 
dissimuar two FeatureVectors are. plates call upon meir associated ContentElements and 
read (istream: InputStream=null): void retrieve the best representation of the element in the context 
Read a FeatureVector from a InputStream (language- 0 f the current delivery environment. A CompositeTemplate 
specific). 35 ensures that given its real estate its subcomponents are 
write (ostream: OutputStream-null): void intelligently laid out with the best-suited media representa- 
Write a FeatureVector to a OutputStream (language- tion of a ContentElement (ContentMediaElement). To fur- 
specific), ther illustrate, an example set of presentation components 
Functional Design are shown in FIG. 29. These example components are not 
Now the corporate web site exemplary embodiment will 40 fully specified, but they illustrate what is expected of a 
be used to discuss the functional aspects of creating, assem- presentation component. 

bling and presenting an application using the architectural In this example presentation model, HTMLDoc, 

framework according to the present invention. HTMLPage, and HTMLBody are CompositeTemplates, 

FIG. 26 shows an exemplary content database for the while HTMLBlock and HTMLAd are non-composite Tern- 
corporate web site according to the present invention. From 45 plates. The semantics of the components are loosely the 
this sparse set of content, a simple portion of a web service following: (1) a HTMLBodyWithAd will always require 1 
will be designed. The database contains content elements HTMLBlock and 1 HTMLAd; (2) AHTMLPage can contain 
with varying representation. In some cases, the content 1 or more components of type HTMLBody; and (3) by 
element has multiple representations of the same type but default, a HTMLDoc contains one HTMLPage. 
with different media characteristics, e.g., the Nature Con- 50 Additionally, a HTMLDoc maps to one Scene object. More 
servation Ad has two image representations but with differ- importantly, if a HTMLDoc determines that one HTMLPage 
ing specification for aspect ratio. The more varied the is insufficient to present a Scene, it may for example, 
database in terms of types of representations and the number dynamically allocate two HTMLPages and mapping to one 
of versions of the same type of representation, the more StoryElement. This last point demonstrates the power and 
contextual delivery of content for the user. 55 flexibility of the architectural framework according to the 

A story model will be described by using the exemplary present invention, if designed and implemented correctly, 

web base service to show how a hierarchical organization of Generally the execution process from a high level view 

filters creates modular highly complex applications that are for assembling a story, generating a presentation that shows 

assembled dynamically, shaped by_the characteristics of the the story, and handling any user events includes: 
current user and the avdlableicontent: eo Creating and initializing a UserModelManager; 

FIG. 27 shows a block diagram of a high level view of a ~ j • r- • 

e , . Creating and initializing a StoryEngine; 

portion of the exemplary web^b ase, service . This portion J ^ 

focuses on the element of the story thatpresents information Creating and initializing a PresentationEngine; 

on the history of the Cedar Fever Bowl. The history of the Selecting an story element (i.e., Scene) to be the initial 

bowl game goes back to 1995 and the intent of an imaginary 65 element of the story; 

web development team is to show a recap of each game Calling upon the StoryEngine to assemble a "story" given 

(1995, 1996) and associate an advertisement alongside each the initial element; 
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Upon the StoryEngine completing its assembly task, 
calling upon the PresentationEngine to present the 
"story"; 

Dispatching a story-relevant event to the StoryEngine to 
determine the next story element (Scene) to play; and 5 

Based on the outcome of the event, set the next story 
element (Scene) to be assembled and subsequently 
presented. 

The following illustrates the assembling of a scene for the 
exemplary web service example, the 1996Game, contained 10 
in the Bowl History Scene. Initially, it is assumed that the 
Story Model at this point has been constructed and mapped 
with the appropriate ContentElements. The StoryEngine 
starts off the assembly of the 1996 Game Scene. Each 
successive Scene calls upon their contained Filters resulting 
in a depth-first traversal of the filter hierarchy. Each Scene 1 
supplies its inputs to its contained Filters. This is the default 
execution behavior, which may be overridden by the appli- 
cation designers implementing their own base Composite- 
Template class that redefines the execution semantics. 

Specifically, the 1996GameReview, a FeatureFilter, is 20 
calling for a content element with the feature, "content_id= 
1996GameReview" which is an explicit call for a specific 
ContentElement. Next, the element named 
PersonalizedAd_2, an AndFilter, is retrieving advertise- 
ments that have the feature "content_Jype-Ad" and that 25 
favorably match the user's UserModel. Having met these 
constraints, the last Filter, a RuleFilter, checks to see if the 
ContentElements that have resulted from the two previous 
Filters (within the AndFilter) are on the StoryEngine's 
already-played list. If all have been "played", one content 30 
element from that current set that has the feature is selected, 
"keyword-sports", otherwise any one ContentElement from 
the current set is chosen. 

The AndFilter feeds the set of ContentElements resulting 
from each contained Filter to the next, which differs from the 35 
Scene^^lter that, simply supplies the same inputs to each 
containe^Eilter. ^Additionally, when the UserFilter is 
executed, it retrieves the user's UserModel via the UMMgr 
(user model manager) to carry out its execution. 

To generate a final presentation of a Scene, it is assumed 40 
that the process of mapping the story elements to the 
presentation elements has already occurred and its outcome 
is shown partially in FIG. 30. The process of rendering 
(actually displaying the presentation) is not shown or 
described. 45 

In generating a presentation, we have a hierarchical 
structure that maps to the hierarchical structure in the 
StoryModel. Once again, the structure is traversed in a 
depth-first manner. Each non-composite template (leaf ele- 
ment in the hierarchy) selects one ContentMediaElement so 
object that represents the template's associated ContentEle- 
ment. Once a CompositeTemplate 's sub-components have 
satisfactorily selected their ContentMediaElements, the 
CompositeTemplate calls upon a LayoutElement to arrange 
the layout of these sub-components. Once a layout is 55 
generated, the CompositeTemplate evaluates the candidate 
presentation based on its criteria as defined in a concrete 
class. If satisfied, control is passed back to its containing 
template, and the whole process starts all over again for the 
siblings in the hierarchy. Eventually, control returns to the 60 
top-level, and if all else evaluated satisfactorily, the overall 
presentation is ready to be rendered. 

In general, the rendering process simply involves travers- 
ing the presentation hierarchy and invoking the show( ) 
operation on the finally selected ContentMediaElement, and 65 
displaying the hierarchy as specified by its CompositeTem- 
plate and its LayoutElement. 
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If any step in the previously described process of evalu- 
ation fails, i.e., a CompositeTemplate is not satisfied with the 
selection of the media representations and/or the layout of its 
subcomponents, the CompositeTemplate then pushes back 
on the contained templates to choose alternate media rep- 
resentations of its associated ContentElement. This process 
involves an iterative generation of the final form of the 
presentation. The final presentation form is shown in FIG. 
31. 

To dispatch a Story-specific event, it is assumed that the 
presentation of a story (i.e., Scene) has been successfully 
rendered by this point. As previously described, ContentEle- 
ments have associated anchors that surface on a Conten- 
tElement because of their relationship with ContentMedi- 
aElements. When an event occurs in the PresentationEngine 
that has relevance to the StoryEngine (i.e., a 
UserSelectionEvent, a TimeoutEvent), the event is for- 
warded to the current Scene. The Scene object decodes the 
event to extract the next Scene to be assembled and pre- 
sented and whole process starts over. 

It is noted that the foregoing examples have been pro- 
vided merely for the purpose of explanation and are in no 
way to be construed as limiting of the present invention. 
While the present invention has been described with refer- 
ence to a preferred embodiment, it is understood that the 
words which have been used herein are words of description 
and illustration, rather than words of limitation. Changes 
may be made within the purview of the appended claims, as 
presently stated and as amended, without departing from the 
scope and spirit of the present invention in its aspects. 
Although the present invention has been described herein 
with reference to particular means, materials, and 
embodiments, the present invention is not intended to be 
limited to the particulars disclosed herein, rather, the present 
invention extends to all functionally equivalent structures, 
methods and uses, such as are within the scope of the 
appended claims. 

What is claimed: 

1. A method for creating and delivering an interactive 
multimedia application configured to dynamically adapt to 
at least one user, the method comprising: 

creating a story engine by the interactive multimedia 
application; 

creating a user model manager by the interactive multi- 
media application; 

providing the story engine with application-specific infor- 
mation and user information; 

providing the story engine with a user model from the user 
model manager, the user model representing interests 
and trends of the at least one user; 

providing the story engine with a narrative structure 
defined by semantics of the interactive multimedia 
application; 

producing user-relevant content by filtering a content 
model, the user model being usable for the filtering; 

creating a presentation engine by the interactive multi- 
media application; 

providing the presentation engine with the narrative 
structure, uninitialized content mode, and a presenta- 
tion model, the uninitialized content model being 
empty; 

generating an abstract presentation defined by the presen- 
tation model, the abstract presentation being generated 
by the presentation engine; 

generating a concrete presentation by using at least heu- 
ristics of the abstract presentation, the concrete presen- 
tation being generated by the presentation engine; and 
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displaying the concrete presentation by the presentation 
engine; 

wherein the abstract presentation and the presentation 
engine autonomously handle interaction scenarios, and 
the interests and trends of the at least user are periodi- 
cally updated based on an interaction history and the 
user model, the interactive multimedia application 
being self-improving and self-sustaining. 

2. The method according to claim 1, the interactive 
multimedia application being created using object-oriented 
design techniques. 

3. The method according to claim 2, the interactive 
multimedia application being using JAVA. 

4. A method for delivering a dynamically adaptive, inter- 
active multimedia application comprising: 

creating a user model manager that generates a user model 
representing at least one of interests and trends of a 
user; 

creating a story engine that receives information specific 
to the multimedia application and the user model, tbe 
story engine generating a story based on at least ooe of 
the multimedia application information and the user 
model; 

creating a presentation engine that receives the story from 
the story engine and a presentation model, the presen- 
tation engine generating an abstract presentation, 
defined by at least the presentation model, and a 
concrete presentation using at least heuristics of the 
abstract presentation; and 

displaying the concrete presentation. 

5. The method for delivering a dynamically adaptive, 
interactive multimedia application according to claim 4, 
further comprising: 

receiving user feedback; and 

updating the user model based on the user feedback. 

6. The method for delivering a dynamically adaptive, 
interactive multimedia application according to claim 4, 
further comprising: 

receiving user feedback; and 

updating the story based on the user feedback. 
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7. The method for delivering a dynamically adaptive, 
interactive multimedia application according to claim 4, in 
which the multimedia application information comprises at 
least one of a content model and a story model. 

8. A computer readable medium for storing a computer 
program that builds and presents a multimedia application, 
the computer readable medium comprising: 

a user modeling source code segment that creates a user 
model manager for generating at least one user model 
for each user, the at least one user model representing 
interests and trends of the corresponding user; 

a story source code segment that creates a story engine for 
receiving information specific to the multimedia appli- 
cation and the at least one user model, and generating 
a story based on at Least one of the multimedia appli- 
cation information and the user model; 

a presentation source code segment that creates a presen- 
tation engine for generating an abstract presentation, 
based on a presentation model and the story generated 
by the story engine, and generating a concrete presen- 
tation based on the abstract presentation; and 

a display source code segment that enables display of the 
concrete presentation. 

9. The computer readable medium according to claim 8, 
in which the user modeling source code segment creates the 
user model by gathering data from the user; analyzing a use 
history of the user; monitoring data related to the user; and 
detecting patterns of the user. 

10. The computer readable medium according to claim 8, 
in which the information specific to the multimedia appli- 
cation comprises information that is able to be represented 
via at least one media type. 

11. The computer readable medium according to claim 8, 
in which the presentation source code segment accounts for 
a narrative style, a narrative context and a demand of the 
presentation delivery environment 
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